992 research outputs found
Tortoise: Interactive System Configuration Repair
System configuration languages provide powerful abstractions that simplify
managing large-scale, networked systems. Thousands of organizations now use
configuration languages, such as Puppet. However, specifications written in
configuration languages can have bugs and the shell remains the simplest way to
debug a misconfigured system. Unfortunately, it is unsafe to use the shell to
fix problems when a system configuration language is in use: a fix applied from
the shell may cause the system to drift from the state specified by the
configuration language. Thus, despite their advantages, configuration languages
force system administrators to give up the simplicity and familiarity of the
shell.
This paper presents a synthesis-based technique that allows administrators to
use configuration languages and the shell in harmony. Administrators can fix
errors using the shell and the technique automatically repairs the higher-level
specification written in the configuration language. The approach (1) produces
repairs that are consistent with the fix made using the shell; (2) produces
repairs that are maintainable by minimizing edits made to the original
specification; (3) ranks and presents multiple repairs when relevant; and (4)
supports all shells the administrator may wish to use. We implement our
technique for Puppet, a widely used system configuration language, and evaluate
it on a suite of benchmarks under 42 repair scenarios. The top-ranked repair is
selected by humans 76% of the time and the human-equivalent repair is ranked
1.31 on average.Comment: Published version in proceedings of IEEE/ACM International Conference
on Automated Software Engineering (ASE) 201
Automatically Repairing Programs Using Both Tests and Bug Reports
The success of automated program repair (APR) depends significantly on its
ability to localize the defects it is repairing. For fault localization (FL),
APR tools typically use either spectrum-based (SBFL) techniques that use test
executions or information-retrieval-based (IRFL) techniques that use bug
reports. These two approaches often complement each other, patching different
defects. No existing repair tool uses both SBFL and IRFL. We develop RAFL
(Rank-Aggregation-Based Fault Localization), a novel FL approach that combines
multiple FL techniques. We also develop Blues, a new IRFL technique that uses
bug reports, and an unsupervised approach to localize defects. On a dataset of
818 real-world defects, SBIR (combined SBFL and Blues) consistently localizes
more bugs and ranks buggy statements higher than the two underlying techniques.
For example, SBIR correctly identifies a buggy statement as the most suspicious
for 18.1% of the defects, while SBFL does so for 10.9% and Blues for 3.1%. We
extend SimFix, a state-of-the-art APR tool, to use SBIR, SBFL, and Blues.
SimFix using SBIR patches 112 out of the 818 defects; 110 when using SBFL, and
55 when using Blues. The 112 patched defects include 55 defects patched
exclusively using SBFL, 7 patched exclusively using IRFL, 47 patched using both
SBFL and IRFL and 3 new defects. SimFix using Blues significantly outperforms
iFixR, the state-of-the-art IRFL-based APR tool. Overall, SimFix using our FL
techniques patches ten defects no prior tools could patch. By evaluating on a
benchmark of 818 defects, 442 previously unused in APR evaluations, we find
that prior evaluations on the overused Defects4J benchmark have led to overly
generous findings. Our paper is the first to (1) use combined FL for APR, (2)
apply a more rigorous methodology for measuring patch correctness, and (3)
evaluate on the new, substantially larger version of Defects4J.Comment: working pape
Wasm/k: Delimited Continuations for WebAssembly
WebAssembly is designed to be an alternative to JavaScript that is a safe,
portable, and efficient compilation target for a variety of languages. The
performance of high-level languages depends not only on the underlying
performance of WebAssembly, but also on the quality of the generated
WebAssembly code. In this paper, we identify several features of high-level
languages that current approaches can only compile to WebAssembly by generating
complex and inefficient code. We argue that these problems could be addressed
if WebAssembly natively supported first-class continuations. We then present
Wasm/k, which extends WebAssembly with delimited continuations. Wasm/k
introduces no new value types, and thus does not require significant changes to
the WebAssembly type system (validation). Wasm/k is safe, even in the presence
of foreign function calls (e.g., to and from JavaScript). Finally, Wasm/k is
amenable to efficient implementation: we implement Wasm/k as a local change to
Wasmtime, an existing WebAssembly JIT. We evaluate Wasm/k by implementing C/k,
which adds delimited continuations to C/C++. C/k uses Emscripten and its
implementation serves as a case study on how to use Wasm/k in a compiler that
targets WebAssembly. We present several case studies using C/k, and show that
on implementing green threads, it can outperform the state-of-the-art approach
Asyncify with an 18% improvement in performance and a 30% improvement in code
size
Software fault identification via dynamic analysis and machine learning
Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2003.Includes bibliographical references (p. 65-67).I propose a technique that identifies program properties that may indicate errors. The technique generates machine learning models of run-time program properties known to expose faults, and applies these models to program properties of user-written code to classify and rank properties that may lead the user to errors. I evaluate an implementation of the technique, the Fault Invariant Classifier, that demonstrates the efficacy of the error finding technique. The implementation uses dynamic invariant detection to generate program properties. It uses support vector machine and decision tree learning tools to classify those properties. Given a set of properties produced by the program analysis, some of which are indicative of errors, the technique selects a subset of properties that are most likely to reveal an error. The experimental evaluation over 941,000 lines of code, showed that a user must examine only the 2.2 highest-ranked properties for C programs and 1.7 for Java programs to find a fault-revealing property. The technique increases the relevance (the concentration of properties that reveal errors) by a factor of 50 on average for C programs, and 4.8 for Java programs.by Yuriy Brun.M.Eng
Passport: Improving Automated Formal Verification Using Identifiers
Formally verifying system properties is one of the most effective ways of
improving system quality, but its high manual effort requirements often render
it prohibitively expensive. Tools that automate formal verification, by
learning from proof corpora to suggest proofs, have just begun to show their
promise. These tools are effective because of the richness of the data the
proof corpora contain. This richness comes from the stylistic conventions
followed by communities of proof developers, together with the logical systems
beneath proof assistants. However, this richness remains underexploited, with
most work thus far focusing on architecture rather than making the most of the
proof data.
In this paper, we develop Passport, a fully-automated proof-synthesis tool
that systematically explores how to most effectively exploit one aspect of that
proof data: identifiers. Passport enriches a predictive Coq model with three
new encoding mechanisms for identifiers: category vocabulary indexing, subword
sequence modeling, and path elaboration. We compare Passport to three existing
base tools which Passport can enhance: ASTactic, Tac, and Tok. In head-to-head
comparisons, Passport automatically proves 29% more theorems than the
best-performing of these base tools. Combining the three Passport-enhanced
tools automatically proves 38% more theorems than the three base tools
together, without Passport's enhancements. Finally, together, these base tools
and Passport-enhanced tools prove 45% more theorems than the combined base
tools without Passport's enhancements. Overall, our findings suggest that
modeling identifiers can play a significant role in improving proof synthesis,
leading to higher-quality software
Fairness Testing: Testing Software for Discrimination
This paper defines software fairness and discrimination and develops a
testing-based method for measuring if and how much software discriminates,
focusing on causality in discriminatory behavior. Evidence of software
discrimination has been found in modern software systems that recommend
criminal sentences, grant access to financial products, and determine who is
allowed to participate in promotions. Our approach, Themis, generates efficient
test suites to measure discrimination. Given a schema describing valid system
inputs, Themis generates discrimination tests automatically and does not
require an oracle. We evaluate Themis on 20 software systems, 12 of which come
from prior work with explicit focus on avoiding discrimination. We find that
(1) Themis is effective at discovering software discrimination, (2)
state-of-the-art techniques for removing discrimination from algorithms fail in
many situations, at times discriminating against as much as 98% of an input
subdomain, (3) Themis optimizations are effective at producing efficient test
suites for measuring discrimination, and (4) Themis is more efficient on
systems that exhibit more discrimination. We thus demonstrate that fairness
testing is a critical aspect of the software development cycle in domains with
possible discrimination and provide initial tools for measuring software
discrimination.Comment: Sainyam Galhotra, Yuriy Brun, and Alexandra Meliou. 2017. Fairness
Testing: Testing Software for Discrimination. In Proceedings of 2017 11th
Joint Meeting of the European Software Engineering Conference and the ACM
SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE),
Paderborn, Germany, September 4-8, 2017 (ESEC/FSE'17).
https://doi.org/10.1145/3106237.3106277, ESEC/FSE, 201
- …